Search for: All records

In this work, we proposeMiSleep, a deep learning augmented millimeter-wave (mmWave) wireless system to monitor human sleep posture by predicting the 3D location of the body joints of a person during sleep. Unlike existing vision- or wearable-based sleep monitoring systems,MiSleepis not privacy-invasive and does not require users to wear anything on their body.MiSleepleverages knowledge of human anatomical features and deep learning models to solve challenges in existing mmWave devices with low-resolution and aliased imaging, and specularity in signals.MiSleepbuilds the model by learning the relationship between mmWave reflected signals and body postures from thousands of existing samples. Since a practical sleep also involves sudden toss-turns, which could introduce errors in posture prediction,MiSleepdesigns a state machine based on the reflected signals to classify the sleeping states into rest or toss-turn, and predict the posture only during the rest states. We evaluateMiSleepwith real data collected from Commercial-Off-The-Shelf mmWave devices for 8 volunteers of diverse ages, genders, and heights performing different sleep postures. We observe thatMiSleepidentifies the toss-turn events start time and duration within 1.25 s and 1.7 s of the ground truth, respectively, and predicts the 3D location of body joints with a median error of 1.3 cm only and can perform even under the blankets, with accuracy on par with the existing vision-based system, unlocking the potential of mmWave systems for privacy-noninvasive at-home healthcare applications. 

We propose MiShape, a millimeter-wave (mmWave) wireless signal based imaging system that generates high-resolution human silhouettes and predicts 3D locations of body joints. The system can capture human motions in real-time under low light and low-visibility conditions. Unlike existing vision-based motion capture systems, MiShape is privacy non-invasive and can generalize to a wide range of motion tracking applications at-home. To overcome the challenges with low-resolution, specularity, and aliasing in images from Commercial-Off-The-Shelf (COTS) mmWave systems, MiShape designs deep learning models based on conditional Generative Adversarial Networks and incorporates the rules of human biomechanics. We have customized MiShape for gait monitoring, but the model is well adaptive to any tracking applications with limited fine-tuning samples. We experimentally evaluate MiShape with real data collected from a COTS mmWave system for 10 volunteers, with diverse ages, gender, height, and somatotype, performing different poses. Our experimental results demonstrate that MiShape delivers high-resolution silhouettes and accurate body poses on par with an existing vision-based system, and unlocks the potential of mmWave systems, such as 5G home wireless routers, for privacy-noninvasive healthcare applications. 

This paper proposes MilliPose, a system that facilitates full human body silhouette imaging and 3D pose estimation from millimeterwave (mmWave) devices. Unlike existing vision-based motion capture systems, MilliPose is not privacy-invasive and is capable of working under obstructions, poor visibility, and low light conditions. MilliPose leverages machine-learning models based on conditional Generative Adversarial Networks and Recurrent Neural Network to solve the challenges of poor resolution, specularity, and variable reflectivity with existing mmWave imaging systems. Our preliminary results show the efficacy of MilliPose in accurately predicting body joint locations under natural human movement. 

Respiratory diseases, like Asthma, COPD, have been a significant public health challenge over decades. Portable spirometers are effective in continuous monitoring of respiratory syndromes out-of-clinic. However, existing systems are either costly or provide limited information and require extra hardware. In this paper, we present mmFlow, a low-barrier means to perform at-home spirometry tests using 5G smart devices. mmFlow works like regular spirometers, where a user forcibly exhales onto a device; but instead of relying on special-purpose hardware, mmFlow leverages built-in millimeter-wave technology in general-purpose, ubiquitous mobile devices. mmFlow analyzes the tiny vibrations created by the airflow on the device surface and combines wireless signal processing with deep learning to enable a software-only spirometry solution. From empirical evaluations, we find that, when device distance is fixed, mmFlow can predict the spirometry indicators with performance comparable to inclinic spirometers with <5% prediction errors. Besides, mmFlow generalizes well under different environments and human conditions, making it promising for out-of-clinic daily monitoring. 

The rapid evolution of the telehealth industry, accelerated recently by stay-at-home directives, has created a demand for more ubiquitous health-sensing tools. One such tool is the Spirometer. Spirometers have been used in traditional clinics to measure lung capacity (volume) as well as airflow (flow rate) and have wide applicability in the diagnosis of Asthma, COPD, and other pulmonary diseases. In addition, they can be used to diagnose Dyspnea, i.e., shortness of breath, one of the symptoms of the COVID-19 virus. Several spirometers are available commercially for home-use, but they are either costly, cumbersome or provide limited flow information. We propose SpiroMilli, a low-barrier means to performing spirometry at home using the millimeter-wave (mmWave) technology in 5G-and-beyond devices. To perform a test, users will hold the device in front of their mouth, fully inhale, then sharply exhale. The system will then output seven key indicators, e.g., Forced Vital Capacity (FVC), Peak Expiratory Flow (PEF), etc., along with a flow-volume curve. SpiroMilli’s key idea is intuitive: Strong airflow in front of the mmWave antenna creates tiny vibrations, and these vibrations affect the phase of reflected signals from nearby objects. For example, a 79 GHz device (wavelength: 3.79 mm) will register a 5 µm displacement as a 1◦ phase change.